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Many types of visual representations are used in math textbooks but not all of them contain mathematically 
relevant information. Little research directly addresses the effects of different types of representations on 
mathematics performance. Theories offer differing perspectives about how visual representations such as 
illustrations influence student learning. Here, we investigated the effects of diagrams and contextual 
illustrations on trigonometry problem solving. Diagrams helped all students, but the effect of contextual 
illustrations depended on students’ backgrounds. Additionally, not all subgroups of students accurately 
assessed the effect of illustrations on their performance. We emphasize the need to consider how different 
types of visual representations interact with student characteristics and the problem-solving task. 
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Mathematics textbooks use a wide variety of visual representations, including diagrams, tables, graphs, 
decorative images, and photographs. Given students’ frequent use of textbooks and the large number of 
visual representations in these textbooks, understanding how different types of visual representations affect 
problem solving and learning is critical. 

Most research on the effects of visual representation has been conducted using scientific texts. It has 
focused primarily on diagrams and illustrations accompanying expository texts about causal phenomena. 
In such contexts, graphically integrating visual and verbal information is found to be beneficial, as is 
removing irrelevant information (e.g., Mayer, 2009). However, mathematical and scientific problem 
solving differ in many important ways, including different emphases on causality, spatial relationships, 
procedural and conceptual knowledge, and analytic methods. Thus, findings from research on science 
learning may not apply straightforwardly to math. 

Existing research about the effects of visual representations in mathematics is based primarily on 
studies with elementary-age students, and it presents a complex and mixed picture. Some studies suggest 
that contextual illustrations hurt performance for particular subgroups of students (e.g., Berends & van 
Lieshout, 2009). Other studies suggest that decorative illustrations do not affect performance (e.g., Berends 
& van Lieshout), or that certain types of illustrations can benefit performance (e.g., Hegarty & 
Kozhevnikov, 1999; McNeil, Uttal, Jarvin, & Sternberg, 2009). Many studies also suggest that the 
usefulness of visual representations depends on students’ ability levels (e.g., Booth & Koedinger, 2011; 
Berends & van Lieshout). 


Theoretical Frameworks 


In making sense of research on visual representations, two theoretical frameworks are particularly 
relevant: the Cognitive Theory of Multimedia Learning (e.g., Mayer, 2005, 2009) and Cognitive Load 
Theory (e.g., Sweller, 2004, 2005). Both theories address the processing and learning of information 
presented in different formats. 

The Cognitive Theory of Multimedia Learning (e.g., Mayer, 2009) is based on three assumptions: 

(a) a limited capacity for processing information, (b) separate visual and verbal pathways through which 
information enters the cognitive system, and (c) meaningful learning arising from active processing. 
Cognitive Load Theory (e.g., Sweller, 2005) focuses on the cognitive load—the mental effort from the task 
itself, the processing required to integrate new and old material, and the processing required to work with a 
task’s format. Overall, one idea is that the structure of the cognitive system imposes limits on how learners 
select, organize, and integrate information. These approaches have been used to guide instructional design. 
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Two principles derived from these theories are particularly relevant to the research reported here. The 
multimedia effect holds that words and pictures are better than just words (e.g., Butcher, 2006; Mayer & 
Anderson, 1992) based on the assumption of separate visual and verbal channels which can then be 
integrated for deeper learning. The coherence effect captures the performance benefits that occur when 
extraneous or seductive features of the material are eliminated (e.g., Harp & Mayer, 1997, 1998). Adding 
interesting but irrelevant material can overload the visual or verbal pathways or create too much 
extraneous load, thereby disrupting learning (Sweller, 2005). The coherence effect applies to both text and 
visual material. 


Contextualization Perspective 


Another theoretical perspective applicable to the current study focuses on how contextualizing or 
“grounding” math problems in real-world scenarios can help learners (Goldstone & Son, 2005; Koedinger 
& Nathan, 2004). Contextualization is thought to help students build a model of the situation underlying a 
problem. In addition, realistic content or greater familiarity with the content may promote generalization or 
facilitate reasoning because it fosters integrating the current problem with prior knowledge. Some studies 
have suggested that contextualization is more beneficial for simpler problems (Koedinger, Alibali, & 
Nathan, 2008), whereas other studies have suggested that it is more beneficial for difficult problems or 
lower ability students (Walkington, 2012). This body of research has typically involved contextualizing 
problems by adding verbal information to text, but contextualization can also be accomplished through 
accompanying visual representations. 


Current Study 


It remains an open question as to how the multimedia principle, the coherence principle, and the notion 
of contextualization apply to visual representations used in mathematics. The current research involves 
trigonometry problems accompanied by 4 types of visual representations: combining diagram presence (or 
not) with the presence of contextual illustrations (or not). The contextual illustration could add extraneous 
details through the graphics, but it also could ground the problem situation. We use the term contextual 
illustration since the illustrative features correspond to the spatial layout necessary to solve the problem. 
The perspectives discussed above vary in their predictions about which visual representations will be most 
helpful (see Table 1). We consider these effects in terms of student performance and evaluations of the 
problems. 


Table 1: Predictions from Applicable Theoretical Frameworks 


Theoretical Prediction 
. ; eos Visual representations will : : 
Multimedia Principle help > Text by itself will be hardest 
ee Extraneous information Problems with illustrations will be 
Coherence Principle > 
hurts performance harder 
: Ane : ; : 
Multimedia Coherence | Provide visuals but avoid S. Didgeana by iielf wall becasicat 
Principles extraneous information 
Contextualization Illustrations further ground > Problems with illustrations will be 
Perspective problem solving easier. 


Method 
Participants 


Participants were 93 undergraduates, who received credit in introductory psychology for their 
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participation. The majority (63%) had completed middle school math in the United States. Of those who 
had not, most (82%) had their earlier math education in an Asian country. Over two-thirds (69%) intended 
to major in a math or science field. 

Participants were divided into subgroups based on their intended major (math/science field or not) and 
where their previous math education occurred (U.S. or non-U.S.). Students who were math/science majors 
were fairly evenly split into those who were previously educated in the U.S. (n = 34) and those who were 
previously educated outside the U.S. (7 = 30). The vast majority of participants who were not math/science 
majors were educated in the U.S. (n = 25). Only 4 participants previously educated outside the U.S. were 
not math/science majors; this small group was excluded from the analyses reported here. 


Design and Materials 


Each participant received 4 problems based on a 2 (Diagram Presence) x 2 (Illustration Presence) 
within-subjects design, yielding 4 conditions: text alone, diagram alone, illustration alone, and illustration 
with diagram overlay (see Table 2). Condition order was counterbalanced across participants. 


Table 2: First Background Story, Shown for Each Visual Condition 
No Diagram Diagram 


The parks department is putting a statue 
on a base. The statue is some distance 
away, and you are in a helicopter, eye 
level with its top. The angle of depression 
No to the bottom of the statue (i.e., the top of 

Illustration | the base) is 35 degrees. The height of the 
statue is 50 feet. If someone were to 
stretch a string from the bottom of the 
base directly to you, it would be 100 feet 
long. How tall is the base? 


Illustration 


Each of the 4 problems each participant received involved a different cover story. All required 
applying trigonometric relations to overlapping right triangles to solve for an unknown dimension. The 
different stories had varied combinations of sides and angles, such that the solution processes were not 
identical for any two problems. These types of problems were selected as they lend themselves well to 
concrete situations and are at an appropriate difficulty level for undergraduate participants. The order of 
the cover stories (and thus of the mathematical solutions) was held constant across participants. Each 
problem was on its own page, with the text and visual representation (if present) at the top of the page. 
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The illustration corresponded to the problem situation. As shown in Table 2, although it did have 
decorative features, it was also mathematically relevant because it indicated the spatial layout of the 
components of the story problem. 


Procedure 


Participants received a reference handout (with text and equations but no diagrams) of information 
about triangles and trigonometric formulas. The information was available throughout the study, and 
participants were told that not all of it would be needed. Participants worked through each of the four 
problems at their own paces. After completing the problems, they rated how difficult each problem was, 
how clear it was, and how willing they would be to do more problems like it. They assessed these 
characteristics on a 5-point Likert scale. While making these ratings, participants were permitted to look 
back over the problems but not to change any of their answers. Finally, participants completed a 
questionnaire about their attitudes towards mathematics, their math abilities, and their math background. 


Results 


A sizeable proportion of participants answered all or none of the problems correctly, and these rates 
depended on participant subgroup. For instance, 40% of the students who were not math/science majors 
answered no problems correctly and 30% of the math/science majors who received their previous 
education outside of the U.S. answered all the problems correctly. The results below include all 
participants; however, the patterns also hold for the subset of participants who did not perform at floor or 
ceiling (i.e., correctly answered 1—3 of the 4 problems). 


Did Visual Condition Affect Accuracy? 


We analyzed the dichotomous measure of accuracy on each problem using mixed models logistic 
regression (Bates & Maechler, 2009) in R. The best fitting model included the fixed factors of diagram 
presence, illustration presence, educational background / major (henceforth participant subgroup), and the 
interaction between illustration and participant subgroup. We also included participant and cover story as 
random factors; cover story significantly improved the model’s fit (p < .0001). Coefficients and odds for 
the model are reported in Table 3. 


Table 3: Coefficients from Regression Model for Accuracy Ratings 


Estimate . 
Fixed effects: (logit) ne ON eae Sig 
Intercept -0.53 0.68 0.59 -0.78 0.44 
Diagram — no Reference 
Diagram — yes 1.49 0.28 4.42 5.23 <.0001 
Illustration — no Reference 
Illustration — yes 0.996 0.47 21 2.1 0.04 
Subgroup — outside US & math/science major Reference 
Subgroup — US & math/science major -0.39 0.64 0.68 -0.61 0.54 
Subgroup — US & not math/science major -1.15 0.70 0.32 -1.65 0.10 
Illustration — yes x 
Subgroup — US & math/science major ee aon en es aoe 
Illustration — yes x 
Subgroup — US & not math/science major “be gis ee =) ae 


Model: Accuracy ~ DiagramPresence + IllustrationPresence * Subgroup + (1 | CoverStory) + (1 | ID) 
Random effects (Intercepts): Variance of participant = 3.26 Variance of cover story = 0.91 
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Participants performed significantly better on problems with diagrams than without, p < .0001 (see 
Figure 1; SE are corrected via procedure in Morey, 2008, to reflect within-subjects design). This effect 
existed for all three subgroups and did not interact with illustration presence. 

However, the effect of illustration varied across participant subgroups. Participants who received their 
math education outside the U.S. (and were also math/science majors) performed significantly better with 
illustrations (p < .04) than without. This improvement differed significantly (p = .03) from the slightly 
negative effect of illustration on the US subgroup not majoring in math/science. The accuracy level of the 
subgroup who were not math/science majors was significantly lower than the accuracies the other two 
subgroups when there was an illustration (ps < .02), but this pattern did not reach significance when there 
was no illustration. 


no illustration 


§ illustration 


Proportion Correct 
(M +/- SE) 


no diagram diagram no diagram diagram no diagram diagram 
prev ed. outside US & math/ prev ed. in US & prev ed. in US & 
science major math/science major not math/science major 


Figure 1: Average accuracy (+/- SE) 


Did Visual Condition Affect Participants’ Ratings of the Problems? 


We combined participants' ratings of each problem's clarity and difficulty (reverse-coded) as well as 
their ratings of how willing they would be to do more problems similar to those completed. This composite 
measure offered an assessment of a participant’s overall favorability towards a problem type. Correlations 
among the three measures ranged from .34 to .56, ps < .0001. The best fitting mixed effects model for 
participants’ ratings included the fixed factors of diagram presence, illustration presence, and participant 
subgroup. We also included participant and cover story as random factors; cover story significantly 
improved the model’s fit (p < .001). 

As shown in Figure 2, participants viewed problems with diagrams significantly more favorably than 
those without, and they viewed problems with illustrations significantly more favorably than those 
without; respectively, each of these factors improved the fit of the model, y7(1) = 7.79, p = .005 and y7(1) 
= 10.5, p = .001. However, the magnitude of these effects was relatively small. Comparisons of the 
subgroups indicated that participants who were not math/science majors rated the problems significantly 
lower than participants with math/science majors (ts > 2.71), whose subgroups did not differ from one 
another (ft = 1.70). 
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no illustration 


eee ® illustration 
# 
ist 
as) 
ee 
me 
a 
ss 
3 Se) 
a 
a 

1 

no diagram} diagram |nodiagram| diagram j|nodiagram| diagram 
prev ed. outside US & prev ed. in US & prev ed. in US & 
math/science major math/science major | not math/science major 


Figure 2: Average favorability (+/— SE) 


Discussion 


In this study, participants performed more accurately on trigonometry problems with diagrams. The 
effect of illustrations was mixed. Illustrations yielded a slight improvement in performance for students 
who intended a math/science major, but illustrations slightly hurt performance among students who were 
not intending to major in a math- or science-related field. These findings highlight that ability differences 
affect the use of visual representations. 

The multimedia principle predicts that problems with visual representations would be solved more 
successfully than problems presented as text only. This was clearly the case for diagrams. The more mixed 
influence of contextual illustrations can be considered with respect to the coherence principle and the 
contextualization perspective, which make opposite predictions. Indeed, each prediction fit a subset of the 
participants. As predicted by the contextualization perspective, having an illustration benefited 
performance for participants who were math/science majors. In contrast, as was predicted by the coherence 
principle, illustrations hurt performance for those who were not math/science majors. Overall, though, the 
effects of illustration presence were relatively small. 

These findings indicate that the coherence principle, which has been supported in multiple studies 
using science material (see Mayer, 2009), may not apply so straightforwardly in math. However, the 
coherence principle stresses the removal of extraneous details. Not all details are extraneous, and added 
visual details do not necessarily harm everyone's performance. This research used contextual illustrations 
that could have assisted students in mapping the problem content to the visual representation and thus does 
not necessarily contradict the coherence principle. It is also worth noting that the contextualized 
illustrations we used were more relevant to mathematics than the majority of illustrations that are found in 
American mathematics textbooks (Cooper et al., 2012; Mayer et al., 1995). Addressing the impact of 
purely decorative illustrations will be an important extension of this research. 

Focusing on the cognitive load required by these problems offers a possible way to combine the two 
perspectives on the effect of illustrations and understand the dependence of the effect on subgroup. The 
cost of encoding and integrating the extraneous information (such as the design of the base of the statue) 
conveyed in illustrations may outweigh any possible benefits from contextualization if cognitive load 
surpasses the available cognitive resources. Illustrations might be more helpful for individuals with more 
math experience because such individuals can construct a contextualized mental representation of the 
problem scenario without exceeding their available cognitive resources. However, other research on 
contextualization has found grounding problems to offer greater benefits for students of lower math ability 
(see Walkington, 2012). 

Diagram presence increased the favorability with which participants viewed the problems, as did 
illustration presence. Comparing this with performance data indicates that all participants’ metacognitive 
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beliefs about problems with diagrams matched their actual performance. However, only students intending 
a math/science major accurately perceived the effect of illustration presence. Participants who were not 
math/science majors performed the same or worse when an illustration was present, despite their more 
favorable view of these problems. This pattern of findings is particularly important to consider in light of 
the motivation-based argument that textbook visuals will help engage learners, particularly those with low 
math interest (see Durik & Harackiewicz, 2007, for related findings). However, it aligns with the research 
arising from the Cognitive Theory of Multimedia Learning and from Cognitive Load Theory, which hold 
that these extraneous but interesting details can be problematic for learning. As noted above, this may hold 
true especially when an individual’s resources are taxed, which is more likely to occur for individuals with 
lower background knowledge. 

The overall differences we observed in accuracy between students of different backgrounds are not 
surprising in light of the well-documented finding that students from many foreign countries outperform 
American students in math (Fleischman et al., 2010). What is more interesting is that students of different 
backgrounds were differentially affected by visual representations. The underlying constructs tapped by 
our measures of students’ backgrounds need to be characterized with greater precision. We collected data 
on the intended majors and the country in which they received their middle school education. These 
measures may simply reduce to experience and interest in math; however, further research on students’ 
backgrounds and how they affect performance is needed. 

It is also worth noting that overall levels of performance in this study were not high, even in the 
highest performing subgroup. The problems we used were quite complex, and many components needed to 
be performed correctly in order to reach an accurate final answer. Students needed to know how to map 
information from the problem content to the visual representation and from the visual representation to 
their mental representation of the problem. Students also needed to identify what quantity to solve for, 
figure out the steps needed to reach the solution, and correctly apply the trigonometric formulas to reach a 
final answer. Understanding the differential effects of the type of visual representation on these different 
components of problem solving is an important arena for future research (see Butcher, 2006; McNeil et al., 
2009). 

In sum, this work highlights the need for a continued focus on the ways in which visual representations 
support learners’ strategic problem solving and learning. Rather than asking simply which types of 
illustrations serve learners better, it is important to identify how learners with different backgrounds and 
skill levels utilize visual representations when solving problems. 
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